Existing methods of interconnecting a network infrastructure by way of dedicated, high-speed lines, for example for remote Logical Unit (LUN) mirroring, are limited. For example, lines between two replicating disk arrays are constrained to be identical in capacity to attain the best chance for non-ordered writes to be applied to a remote disk array in the correct order. For example, if a user begins with a single Asynchronous Transfer Mode Optical Channel-3 (ATM OC-3) line, with a throughput of about 19 Megabytes/second (MB/s), and adds an identical line, the aggregate mirroring between the disk arrays improves by about 100 percent, within the scaling limits of the hardware/software infrastructure.
However, if a user instead adds a slower second line, for example a T3 line with throughput of about 5 MB/s, the aggregate mirroring throughput, rather than increasing by the addition of resources, actually has an aggregate mirroring throughput that is reduced to approximately twice the speed of the slower line. In a configuration with multiple lines, the throughput is reduced to the speed of the slowest line times the number of lines. Throughput reduction results from quirks of a customary ‘round robin’ selection process for determining next usage of a transmission line.
Throttling occurs for aggregation of transmission lines of dissimilar speeds. Throttling phenomena can be explained with analogy to a group of cars lined up on a first-come, first-served basis on one side of a river awaiting usage of two single-car ferries to transport the cars to the river's other side. The number of cars that can be ferried per hour via the slowest ferry is given to be X. If both ferries are the same speed, the number of cars ferried per hour is 2X. In this case, requirements that the cars reach the other side in the exact order of queuing on the first side, and that usage of the ferries alternates in a strictly round robin order, do not impact the throughput of either ferry.
In contrast, if one ferry travels at a speed Y that is substantially faster than X, imposition of a requirement of round robin ferry selection can greatly limit the number of cars ferried in comparison to the maximum possible number that the faster speed Y could otherwise enable. Specifically, even with a ferry capable of traveling at the faster speed Y, strict adherence to round robin ferrying limits the capacity of the aggregate transport system to a speed of 2X. The higher speed of the faster ferry does not improve aggregate capacity because the faster ferry is nearly always stalled waiting for the slower ferry. When the speed Y is much larger than the speed X, the potential traffic, analogous to link bandwidth, that is forfeited by usage of round robin selection is equal to Y−X.
With respect to interconnections in a storage system, round robin link selection similarly impacts throughput. For example, if a T3 line with capacity of 5 MB/s is currently in use between two mirroring disk arrays and Information Technology (IT) infrastructure changes add an ATM OC-3 line with capacity of 20 MB/s, the total aggregate throughput is limited to 2×5 MB/s or 10 MB/s due to throttling effects of forced round robin link usage. Conversely, if a 20 MB/s line is originally in use and a newly available 5 MB/s line is added, the result is a drop in aggregate line performance from 20 MB/s to 10 MB/s. FIGS. 8A, 8B, 8C, and 8D show examples of detrimental throttling due to round robin usage of unmatched links. In each case a pair of disk arrays 800 communicates over a variety of links. FIG. 8A depicts a system connected by a 100 MB/s Fibre Channel (FC) link and a shared FC link with available capacity of 20 MB/s. The potential aggregate capacity is 200 MB/s but the actual aggregate throughput is 40 MB/s, twice the available capacity from the shared link.
FIG. 8B shows a system connected by a dedicated 100 MB/s internet Small Compute Systems Interface (iSCSI)/Ethernet link and a shared iSCSI/Ethernet link with an available capacity of 50 MB/s. The potential aggregate capacity is 200 MB/s but the actual aggregate throughput is 100 MB/s, twice the available capacity from the shared link.
FIG. 8C illustrates a system connected by a dedicated 100 MB/s Fibre Channel (FC) link and a 17 MB/s Enterprise System Connection (ESCON) link. The potential aggregate capacity is 134 MB/s but the actual aggregate throughput is 34 MB/s, twice the slow ESCON link.
FIG. 8D illustrates a system connected by two dedicated 100 MB/s Gigabit Ethernet (gigE) links and a dedicated 10 MB/s Ethernet 10/100 bT link. The potential aggregate capacity is 210 MB/s but the actual aggregate throughput is 30 MB/s, three times the throughput of the slowest link.